Rényi information dimension: fundamental limits of almost lossless analog compression
نویسندگان
چکیده
In Shannon theory, lossless source coding deals with the optimal compression of discrete sources. Compressed sensing is a lossless coding strategy for analog sources by means of multiplication by real-valued matrices. In this paper we study almost lossless analog compression for analog memoryless sources in an information-theoretic framework, in which the compressor or decompressor is constrained by various regularity conditions, in particular linearity of the compressor and Lipschitz continuity of the decompressor. The fundamental limit is shown to be the information dimension proposed by Rényi in 1959.
منابع مشابه
Lossless Analog Compression
We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary random vectors x ∈ R from the noiseless linear measurements y = Ax with measurement matrix A ∈ Rn×m. Our theory is inspired by the groundbreaking work of Wu and Verdú (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers zero err...
متن کاملLossless Data Compression at Finite Blocklengths
This paper provides an extensive study of the behavior of the best achievable rate (and other related fundamental limits) in variable-length lossless compression. In the non-asymptotic regime, the fundamental limits of fixed-to-variable lossless compression with and without prefix constraints are shown to be tightly coupled. Several precise, quantitative bounds are derived, connecting the distr...
متن کاملMinimax Rényi redundancy
The redundancy for universal lossless compression in Campbell’s setting is characterized as a minimax Rényi divergence, which is shown to be equal to the maximal α-mutual information via a generalized redundancy-capacity theorem. Special attention is placed on the analysis of the asymptotics of minimax Rényi divergence, which is determined up to a term vanishing in blocklength.
متن کاملA suboptimal lossy data compression based on approximate pattern matching
Wojciech Szpankowski§ Department of Computer Science Purdue University W. Lafayette, IN 47907 U.S.A. [email protected] A practical suboptimal (variable source coding) algorithm for lossy data compression is presented. This scheme is based on approximate string matching, and it naturally extends the lossless Lempel-Ziv data compression scheme. Among others we consider the typical length of appro...
متن کاملUniversal Weak Variable-Length Source Coding on Countable Infinite Alphabets
Motivated from the fact that universal source coding on countably infinite alphabets is not feasible, this work introduces the notion of “almost lossless source coding”. Analog to the weak variable-length source coding problem studied by Han [3], almost lossless source coding aims at relaxing the lossless block-wise assumption to allow an average per-letter distortion that vanishes asymptotical...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- IEEE Trans. Information Theory
دوره 56 شماره
صفحات -
تاریخ انتشار 2010